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Abstract  This  paper  shows  that  the  solutions  to  various  convex  l\  minimization  problems  are  unique  if  and 
only  if  a  common  set  of  conditions  are  satisfied.  This  result  applies  broadly  to  the  basis  pursuit  model,  basis 
pursuit  denoising  model,  Lasso  model,  as  well  as  other  l\  models  that  either  minimize  f(Ax  —  b )  or  impose 
the  constraint  f(Ax  —  b)  <  cr,  where  /  is  a  strictly  convex  function.  For  these  models,  this  paper  proves 
that,  given  a  solution  x*  and  defining  I  =  supp(:r*)  and  s  =  sign(a;J),  x*  is  the  unique  solution  if  and  only 
if  Aj  has  full  column  rank  and  there  exists  y  such  that  Ajy  =  s  and  \aJy\oo  <  1  for  i  (jL  I.  This  condition 
is  previously  known  to  be  sufficient  for  the  basis  pursuit  model  to  have  a  unique  solution  supported  on  I. 
Indeed,  it  is  also  necessary,  and  applies  to  a  variety  of  other  models.  The  paper  also  discusses  ways  to 
recognize  unique  solutions  and  verify  the  uniqueness  conditions  numerically. 

Keywords  £i  minimization  •  basis  pursuit  •  LASSO  •  solution  uniqueness  •  strict  complementarity 


1  Introduction 

Let  x  £  R”  be  the  decision  variable.  This  paper  studies  the  unique  solutions  of  the  t\  minimization  problems 
including  the  basis  pursuit  problem  [8] 


min||x||i  s.t.  Ax  =  b  (la) 
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and  convex  problems 


min  fi(Ax  —  b)  +  A||x||i,  (lb) 

min  ||a;||i,  s.t.  —  b)  <  a,  (lc) 

min  fs{Ax  —  b),  s.t.  ||a;||i  <  r,  (Id) 

where  A,  a,  t  >  0  are  scalar  parameters,  A  is  a  matrix,  and  fi(x),i  =  1,2,3  are  strictly  convex  functions. 
The  Lasso  problem  [21]  is  a  special  case  of  problem  (lb)  or  (Id)  while  the  basis  pursuit  denoising  problem 
[8]  is  a  special  case  of  problem  (lc)  all  with  /*(•)  =  ^  II  ‘  Hi)  *  =  1,2,3. 

There  is  a  rich  literature  on  analyzing,  solving,  and  applying  problems  in  forms  of  (la)-(ld)  in  the 
communities  of  information  theory,  signal  processing,  statistics,  machine  learning,  optimization,  and  so  on. 
In  many  cases,  problems  (la)-(ld)  need  to  have  unique  solutions;  when  there  are  more  than  one  solution, 
the  set  of  solutions  is  a  convex  set  including  an  infinite  number  of  solutions.  In  compressive  sensing  signal 
recovery,  having  non-unique  solutions  means  that  the  underlying  signal  can  not  be  reliably  recovered  from  the 
given  data.  When  the  basis  pursuit  denoising  problem  has  a  unique  solution,  the  error  between  the  solution 
and  original  sparse  signal  can  be  bounded  by  the  noise  size  [18].  In  feature  selection,  non-unique  solutions 
cause  ambiguity  for  feature  identification.  Even  certain  optimization  methods  and  algorithms,  especially 
those  producing  the  solution  path  of  (lb)-(ld)  over  varying  parameters  such  as  LARS  [14]  and  parametric 
quadratic  programming  [1],  require  solution  uniqueness;  they  will  fail  (or  need  special  treatments)  upon 
encountering  non-unique  solutions.  Therefore,  establishing  a  condition  of  solution  uniqueness  is  important  for 
both  the  analysis  and  computation  of  problems  (la)-(ld).  Fortunately,  there  are  various  sufficient  conditions 
guaranteeing  solution  uniqueness  in  problem  (la),  such  as  Spark  [10,2],  the  mutual  incoherence  condition 
[11,15],  the  null-space  property  (NSP)  [12,9],  the  restricted  isometry  principle  (RIP)  [3],  the  spherical  section 
property  [24],  the  “RIPless”  property  [5],  and  so  on.  Some  conditions  guarantee  the  unique  recovery  of  a 
given  solution  or  solutions  with  a  given  set  of  signs;  other  conditions  provide  the  guarantees  for  all  solutions 
with  sufficiently  few  nonzero  entries.  However,  none  of  them  is  known  to  be  both  necessary  and  sufficient  for 
solution  uniqueness  in  problems  (lb)-(ld).  This  paper  shows  that  given  a  solution  x*  to  any  problem  among 
(la)-(ld),  Condition  1  below  is  both  necessary  and  sufficient  for  x*  to  be  the  unique  solution.  Hence,  it  is 
weaker  than  the  sufficient  conditions  listed  above. 

We  let  X,  X\,  Ya,  and  ZT  denote  the  sets  of  solutions  to  problems  (la)-(ld),  respectively.  We  let  a,;  be 
the  itli  column  of  A  and  Xi  be  the  itli  entry  of  x.  Given  an  index  set  /,  we  frequently  use  Aj  as  the  submatrix 
of  A  formed  by  its  columns  i  €  I  and  xi  as  the  subvector  of  x  formed  by  entries  a i  g  I. 

Our  analysis  makes  the  following  assumptions: 


Assumption  1  Matrix  A  has  full  row  rank. 


Assumption  2  The  solution  sets  X,  X\,  Y„,  and  ZT  of  problems  (la)-(ld),  respectively,  are  nonempty. 


Assumption  3  In  problems  (lb)-(ld),  functions  /i,/2,/3  are  strictly  convex.  In  addition,  the  constraint 
of  problem  (Id)  is  bounding,  namely,  r  <  inf{||x||i  :  fz(Ax  —  b)  =  f£},  where  /|  :=  minygRn  f(Ay  —  b). 


Assumptions  1  and  2  are  standard.  If  Assumption  1  does  not  hold  and  Ax  =  5  is  consistent,  the  problems 


can  be  simplified;  specifically,  one  can  decompose  A  = 


A\ 

Ai 


and  b  = 


so  that  and  A\  has  full  row  rank 


equal  to  rank(A),  and  one  can  replace  the  constraints  Ax  =  b  by  A^x  =  b±  and  introduce  functions  fi  so  that 
fi(Aix  —  bi)  =  fi{Ax  —  b),i  =  1,  2,  3.  Assumption  2  guarantees  that  the  solutions  of  problems  (la)-(ld)  can 
be  attained  so  the  discussion  of  solution  uniqueness  makes  sense.  The  strict  convexity  of  f\,  $2,  fo  and  the 
restriction  on  r  in  Assumption  3  are  also  fairly  basic  toward  solution  uniqueness.  Strict  convexity  rules  out 
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piece-wise  linearity.  (Note  that  /i,/2,/3  can  still  be  non-differentiable.)  If  the  restriction  on  r  is  removed, 
the  solution  uniqueness  of  problem  (Id)  becomes  solely  up  to  f3{Ax  —  b ),  independent  of  ||x||i. 

For  a  given  vector  x* ,  the  following  conditions  on  matrix  A  is  the  key  to  solution  uniqueness,  and  its 
sufficiency  has  been  established  in  [16]. 

Condition  1  Under  the  definitions  I  :=  supp(x*)  C  {l,...,n}  and  s  :=  sign(a;J),  matrix  A  G  Rmxrl  ]ms 
the  following  properties: 

1.  submatrix  Aj  has  full  column  rank ,  and 

2.  there  is  y  €  Rm  obeying  Ajy  =  s  and  HA/LyHoo  <  1- 

The  main  theorem  of  this  paper  asserts  that  Condition  1  is  both  necessary  and  sufficient  to  the  uniqueness 
of  solution  x*. 

Theorem  1  (Solution  uniqueness)  Under  Assumptions  1-3,  given  that  x*  is  a  solution  to  problem  (la), 
(lb),  (lc),  or  (Id),  x*  is  the  unique  solution  if  and  only  if  Condition  1  holds. 

In  addition,  combining  Theorem  1  with  the  optimality  conditions  for  problems  (la)-(ld),  the  following 
theorems  give  the  necessary  and  sufficient  conditions  of  unique  optimality  for  those  problems. 

Theorem  2  (Basis  pursuit  unique  optimality)  Under  Assumptions  1-2,  x*  €  R"  is  the  unique  solution 
to  problem  (la)  if  and  only  if  Ax*  =  b  and  Condition  1  is  satisfied. 

Theorem  3  (Problems  (lb)— (Id)  unique  optimality)  Under  Assumptions  1-3  and  the  additional  as¬ 
sumption  /i,/2,/3  G  C1,  x*  G  Rn  is  the  unique  solution  to  problem  (lb),  (lc),  or  (Id)  if  and  only  if, 
respectively, 

3  p*  G  a||** || i,9  p*  +  XAT\7f1(Ax*  -b)  =  0,  (2a) 

f{Ax*  —  b)  <  a  and  3  p*  G  c?||a:*||i,  rj  >  0, 

9p*  +  r]ATX7 /2(Ae*  —  b)  =  0,  or  (2b) 

||x*||i  <  r  and  3  p*  G  c?||a:*||i,  i/  >  0, 

9  vp*  +  ATX7f3(Ax*  -b)  =  0,  (2c) 

and  in  addition  Condition  1  holds. 

The  proofs  of  these  theorems  are  given  in  Section  2  below. 


1.1  Related  works 

Since  the  sufficiency  is  not  the  focus  of  this  paper,  we  do  not  go  into  more  details  of  the  sufficient  conditions 
that  have  been  mentioned  above.  We  just  point  out  that  several  papers  such  as  [16,6]  construct  the  least- 
squares  (i.e.,  minimal  t^-norm)  solution  y  of  Ajy  =  s  and  establish  sufficient  conditions  for  ||A|’cy||0o  <  1  to 
hold.  Next,  we  review  the  existing  results  on  necessary  conditions  for  the  uniqueness  of  l\  minimizer. 

Work  [4]  considers  problem  (la)  with  complex- valued  quantities  and  A  equal  to  a  clown-sampled  discrete 
Fourier  operator,  for  which  it  establishes  both  the  necessity  and  sufficiency  of  Condition  1  to  the  solution 
uniqueness  of  (la).  Their  proof  uses  the  Hahn-Banach  separation  theorem  and  the  Parseval  formula.  Work 
[23]  lets  the  entries  of  matrix  A  and  vector  x  in  problem  (la)  have  complex  values  and  gives  a  sufficient 
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condition  for  its  solution  uniqueness.  In  regularization  theory,  Condition  1  is  used  to  derive  linear  error 
bounds  under  the  name  of  range  or  source  conditions  in  [18],  which  shows  the  necessity  and  sufficiency  of 
Condition  1  for  solution  uniqueness  of  (la)  in  a  Hilbert-space  setting.  More  recently,  [13]  constructs  the  set 

F  =  {x  :  \\A7jCAj(A7jAJy1sign(xj)\\00  <  1  and  rank (Aj)  =  \  J\}, 


where  J  =  supp(x),  and  then  states  that  the  set  of  vectors  that  can  be  recovered  by  problem  (la)  is  exactly 
characterized  by  the  closure  of  F  if  the  measurement  matrix  A  satisfies  the  so-called  general  position  (GP) 
condition,  namely,  for  any  sign  vector  s  £  {—1,1}”,  the  set  of  columns  {A1}  of  A  £  Rmxn  satisfying  that 
any  fc-dimensional  affine  subspace  of  Rm,  k  <  m,  contains  at  most  k  +  1  points  from  the  set  {s,AP}.  This 
paper  claims  that  the  result  holds  without  the  GP  condition. 

To  our  knowledge,  there  are  few  conditions  addressing  the  solution  uniqueness  of  problems  (lb)-(ld). 
The  following  conditions  in  [16],  [17]  are  sufficient  for  x*  to  the  unique  minimizer  of  (lb)  for  /i(-)  =  |||  •  H^: 


Aj(b-  A/a:})  =  A  •  sign(a;}) , 
WAjcib-A^j) IU  <  A, 

A i  has  full  column  rank. 


(3a) 

(3b) 

(3c) 


However,  they  are  not  necessary  as  demonstrated  by  the  following  example.  Let 


A  = 

1 

0  2 

,  b  = 

1 

0 

2  -2 

1 

and  consider  solving  the  Lasso  problem,  which  is  a  special  case  of  problem  (lb): 

min  ^|| Ax  -  6|||  +  A||a:||i. 


(4) 


(5) 


One  gets  the  unique  solution  x*  =  [0  1/4  0]T  and  /  =  supp(a:*)  =  {2}.  However,  the  inequality  in  condition 
(3b)  holds  with  equality.  In  general,  conditions  (3)  becomes  necessary  in  case  Aj  happens  to  be  a  full  rank 
square  matrix.  This  assumption,  however,  does  not  apply  to  a  sparse  solution  x*.  Nevertheless,  we  summarize 
the  result  in  the  following  corollary,  whose  proof  is  given  at  the  end  of  Section  2. 


Corollary  1  If  x*  is  the  unique  minimizer  of  problem  (lb)  with  /i(-)  =  IIHH  and  if  Aj,  where  I  =  supp(x*), 
is  a  square  matrix  with  full  rank,  then  condition  (3)  holds. 


Very  recently,  work  [22]  investigates  the  solution  uniqueness  of  (5)  and  presents  the  following  result. 

Theorem  4  ([22])  Let  x*  be  a  solution  of  (5)  and  J  :=  {z  :  | af  (b  —  Ax*)\  =  A}.  If  submatrix  Aj  is  full 
column  rank,  then  x*  is  unique.  Conversely,  for  almost  every  b  £  if  x*  is  unique,  then  Aj  is  full  column 
rank. 


In  Theorem  4,  the  necessity  part  “for  almost  every  6”  is  new.  Indeed,  it  is  not  for  every  b.  An  example  is 
given  in  (4)  with  a  unique  solution  x*  and  J  =  {1,2,3},  but  Aj  does  not  full  column  rank.  On  the  other 
hand,  we  can  figure  out  a  special  case  in  which  the  full  column  rankness  of  Aj  becomes  necessary  for  all  b 
in  the  following  corollary,  whose  proof  is  given  at  the  end  of  Section  2. 

Corollary  2  Let  x*  be  a  solution  of  problem  (5)  and  define  I  :=  supp(x*)  and  J  :=  {z  :  \af(b  —  Ax*)\  =  A}. 
If  |  J\  =  |/|  +  1,  then  x*  is  the  unique  solution  if  and  only  if  Aj  has  full  column  rank. 
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2  Proofs  of  Theorems  1—3 

We  establish  Theorem  1  in  three  steps.  Our  first  step  proves  the  theorem  for  problem  (la)  only.  Since  the 
only  difference  between  this  part  and  Theorem  2  is  the  conditions  Ax*  =  &,  we  prove  Theorem  2  first.  In  the 
second  step,  for  problems  (lb)-(ld),  we  show  that  both  ||ar|| i  and  Ax  —  b  are  constant  for  x  over  the  solution 
sets  X\,Ya,  ZT,  respectively.  Finally,  we  prove  Theorem  1  for  problems  (lb)-(ld). 

Proof  (Proof  of  Theorem  2)  We  frequently  use  the  notions  I  =  supp(x*)  and  s  =  sign(xj)  below. 

,  This  part  has  been  shown  in  [16,23].  For  completeness,  we  give  a  proof.  Let  y  satisfy  Condition 
1,  part  2,  and  let  x  £  R"  be  an  arbitrary  vector  satisfying  Ax  =  b  and  x  ^  x*.  We  shall  show  ||x*||i  <  ||x||i. 

Since  Aj  has  full  column  rank  and  x  y^  x* ,  we  have  supp(x)  y^  /;  otherwise  from  A/xJ  =  b  =  AjXi,  we 
would  get  xj  =  xj  and  thus  the  contradiction  x*  =  x. 

From  supp(x)  yf  /,  we  get  bT y  <  ||x||i.  To  see  this,  let  J  :=  supp(x)  \  I ,  which  is  a  non-empty  subset  of 
Ic.  From  Condition  1,  we  have  ||-<4/2/||oo  =  1  and  H-Ajj/Hoo  <  1,  and  thus 

(xi,Ajy)  <  ||xj||i  •  PfylU  <  ||a?z||i, 

{xj,ATjy)  <  ||xj||i  •  Pjy||oo  <  |M|i, 

(the  last  inequality  is  “<”  not  “<”)  which  lead  to 

bTy  =  (x,  ATy)  =  (xj,  Ajy)  +  (xj,  ATjy) 

<IMIi  +  IM1  =  \\x\\i. 


On  the  other  hand,  we  have 

||x*||i  =  (xj,sign(xj))  =  (x*j,  Aj y)  =  (AIx*I,y)  =  bTy 
and  thus  ||x*||i  =  bTy  <  ||x||i. 

Assume  that  x*  is  the  unique  solution  to  (la).  Obviously,  Ax*  =  b. 

It  is  easy  to  obtain  Condition  1,  part  1.  Suppose  it  does  not  hold.  Then,  A/  has  a  nontrivial  null  space, 
and  perturbing  x\  along  the  null  space  will  change  the  objective  ||xj||i  =  sT x\  while  maintaining  A^x}  =  6; 
hence,  this  perturbing  breaks  the  unique  optimality  of  x* .  (In  more  details,  there  exists  a  nonzero  vector 
d  £  R™  such  that  Ajdj  =  0  and  d/c  =  0.  For  any  scalar  a  near  zero,  we  have  sign(xj  +  adi)  =  sign(xj)  =  s 
and  thus  ||x*  +  ad||i  =  sT(x*I  +  adj)  =  sTx*I  +  a(sTdi)  =  ||x*||i  +  a(sTdi).  Since  x*  is  the  unique  solution, 
we  must  have  ||xj  +  ad||i  >  ||x*||i  or,  equivalently,  a(sTdj)  >  0  whenever  a/0.  This  is  impossible  as  we 
can  perturb  a  around  0  both  ways.) 

It  remains  to  construct  a  vector  y  for  Condition  1,  part  2.  Our  construction  is  based  on  the  strong 
convexity  relation  between  a  linear  program  (called  the  primal  problem)  and  its  dual  problem,  namely,  if  one 
problem  has  a  solution,  so  does  the  other,  and  the  two  solutions  must  give  the  same  objective  value.  (For  the 
interested  reader,  this  result  follows  from  the  Hahn-Banach  separation  theorem,  also  from  the  theorem  of 
alternatives  [7].  Alternatively,  it  can  be  obtain  constructively  via  the  Simplex  method;  specifically,  whenever 
a  primal  solution  exists,  the  Simplex  method  terminates  in  a  finite  number  of  steps  with  a  primal-dual 
solution  pair.) 

The  strong  duality  relation  holds  between  (la)  and  its  dual  problem 

max  bTp  s.t.  IjA^Hoo  <  1  (6) 
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because  (la)  and  (6),  as  a  primal-dual  pair,  are  equivalent  to  the  primal-dual  linear  programs 

min  1  Tu  +  l1  v  s.t.  Au  —  Av  =  6,  u  >  0,  v  >  0,  (7a) 

ii,'ueRn 

max  bT q  s.t.  —  1  <  ATq  <  1.  (7b) 

gSRm 

respectively,  where  the  strong  duality  relation  holds  between  (7a)  and  (7b).  By  “equivalent”,  we  mean  that 
one  can  obtain  solutions  from  each  other  by  the  rules: 

given  u*,v*,  obtain  x*  =  u*  —  v* 

given  x*,  obtain  u*  =  max(x*,0),  v*  =  max(— £*,0), 
given  q* ,  obtain  p*  =  q* , 
given  p* ,  obtain  q*  =  p* . 

Therefore,  since  (la)  has  solution  x* ,  there  exists  a  solution  y*  to  (6),  which  satisfies  ||x*||i  =  bT y*  and 
\\ATy*\\00  <  1.  (One  can  obtain  such  y*  from  the  Hahn-Banach  separation  theorem  or  the  theorem  of  alter¬ 
natives  rather  directly.)  However,  y*  may  not  obey  ||Afcy*||00  <  1.  We  shall  perturb  y*  so  that  ||Afc2/*||oo  <  1- 
To  prepare  for  the  perturbation,  we  let  L  :=  {i  G  Ic  :  afy*  =  —1}  and  U  :=  {i  €  Ic  :  afy*  =  1}.  Our 
goal  is  to  perturb  y*  so  that  —  1  <  afy*  <  1  for  i  G  L  U  U  and  y*  remains  optimal  to  (6).  To  this  end, 
consider  for  a  fixed  a  >  0  and  t  :=  ||a;*||i,  the  linear  program 

min  >  ax*—?  aXi,  s.t.  Ax  =  b,  llxlli  <  t.  (8) 

x(zKn  ^  ' 

ieL  ieu 

Since  x*  is  the  unique  solution  to  (la),  it  is  the  unique  feasible  solution  to  problem  (8),  so  problem  (8)  has 
the  optimal  objective  value  Y^ieu  axi  ~  Yi^L  axi  =  0-  By  setting  up  equivalent  linear  programs  like  what 
has  been  done  for  (la)  and  (6)  above,  the  strong  duality  relation  holds  between  (8)  and  its  dual  problem 

max  bTp  —  tq,  s.t.  \\ATp  —  arlloo  <  q,  q  >  0,  (9) 

peRm,<?eR 

where  r  G  R"  is  given  by 

!1>  *  G  L, 

-1,  i  G  U, 

0,  otherwise. 

Therefore,  (9)  has  a  solution  ( p*,q *)  satisfying  bTp*  —  tq*  =  0. 

According  to  the  last  constraint  of  (9),  we  have  q*  >  0,  which  we  split  into  two  cases:  q*  =  0  and  q*  >  0. 

i)  If  q*  =  0,  we  have  ATp*  =  car  and  bTp*  =  0. 

ii)  If  q*  >  0,  we  let  r*  :=  p* /q*,  which  satisfies  bTr*  =  t  =  ||x*||i  and  ||ATr*  —  ^^Hoo  <  1,  or  equivalently, 
-1  +  4r  <  ATr*  <  1  +  4r. 

q  —  ~  q 

Now  we  perturb  y*.  Solve  (9)  with  a  sufficiently  small  a  >  0  and  obtain  a  solution  (p*,q*).  If  case  i)  occurs, 
we  let  y*  <—  y*  +  p*-,  otherwise,  we  let  y*  <—  \{y*  +  r*).  In  both  cases, 

-  bTy*  is  unchanged,  still  equal  to  ||x*||i; 

—  —  1  <  afy*  <  1  holds  for  i  G  L  U  U  after  the  perturbation; 

—  for  each  *  ^  LU  U,  if  ajy*  G  [—1, 1]  or  aj y*  G  (—1, 1)  holds  before  the  perturbation,  the  same  holds  after 
the  perturbation; 

Therefore,  after  the  perturbation,  y*  satisfies: 

1)  bTy*=  ||x*||i, 
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2)  mJVlIoo  <  1,  and 

3)  Pfcy*||oo<l- 

From  1)  and  2)  it  follows 

4)  AJy  =  sign(x,) 

since  ||a:f ||i  =  ||x*||i  =  bTy*  =  (Aj x*I:y*)  =  ( x*j,Afy *)  <  ||af)||i||ATy*||00  <  ||a;J||i  and  thus  (. x},Ajy *)  = 
||arj|| i,  which  dictates  4).  From  3)  and  4),  Condition  1,  part  2,  holds  with  y  =  y*. 

Proof  (Proof  of  Theorem  1  for  problem  (la) )  The  above  proof  for  Theorem  2  also  serves  the  proof  of  Theorem 
1  for  problem  (la)  since  Ax*  =  b  is  involved  only  in  the  optimality  part,  not  the  uniqueness  part. 

Next,  we  show  that  Ax  —  b  is  constant  for  x  over  X\ ,  Y„.  ZT,  and  we  first  prove  a  simple  lemma. 

Lemma  1  Let  f  be  a  strictly  convex  function.  If  f(Ax  —  b)  +  ||x||i  is  constant  on  a  convex  set  S,  then  both 
Ax  —  b  and  ||x||i  are  constant  on  S. 

Proof  It  suffices  to  prove  the  case  where  S  has  more  one  point.  Let  Xi  and  X2  be  any  two  different  points 
in  S.  Consider  the  line  segment  L  connecting  Xi  and  X2 ■  Since  X  is  convex,  we  have  L  C  X  and  that 
/(Ax  —  b)  +  ||x||i  is  constant  on  L.  On  one  hand,  ||x||i  is  piece-wise  linear  over  L\  on  the  other  hand,  the 
strict  convexity  of  /  makes  it  impossible  for  f(Ax  —  b )  to  be  piece-wise  linear  over  L  unless  Ax i  —  b=  Ax 2  —  b. 
Hence,  we  have  Ax  1  —  b  =  Ax 2  —  b  and  thus  f(Ax  1  —  b)  =  f(Ax 2  —  b),  from  which  it  follows  ||xi||i  =  ||x2 1|  1  - 
Since  Xi  and  X2  are  arbitrary  two  points  in  S,  the  lemma  is  proved. 

With  Lemma  1  we  can  show 

Lemma  2  Under  Assumptions  2  and  3,  the  following  statements  for  problems  (lb) -(Id)  hold 

1 )  X\ ,  Y„  and  ZT  are  convex; 

2)  In  problem  (lb),  Ax  —  b  and  ||x||i  are  constant  for  all  x  £  X\; 

3)  Part  2)  holds  for  problem  (lc)  and  Ya; 

f)  Part  2)  holds  for  problem  (Id)  and  ZT. 

Proof  Assumption  2  makes  sure  that  X\ ,Ya,  ZT  are  all  non-empty. 

1)  As  a  well-known  result,  the  set  of  solutions  of  a  convex  program  is  convex. 

2)  Since  fi(Ax  —  b)  +  A||x||i  is  constant  over  x  €  X\  and  /1  is  strictly  convex  by  Assumption  3,  the  result 
follows  directly  from  Lemma  1. 

3)  If  0  e  Ya,  then  the  optimal  objective  is  ||0||i  =  0;  hence,  Y\  =  {0}  and  the  results  hold  trivially. 
Suppose  0  ^  Ya.  Since  the  optimal  objective  ||x||i  is  constant  for  all  x  €  Ya  and  /2  is  strictly  convex  by 
Assumption  3,  to  prove  this  part  in  light  of  Lemma  1,  we  shall  show  /2(Ax  —  b)  =  a  for  all  x  £  Ya. 

Assume  that  there  is  x  £  Ya  such  that  /2(Ax  —  b)  <  a.  Since  /2 ( Ax  —  b)  is  convex  and  thus  continuous  in 
x,  there  exists  a  non-empty  ball  B  centered  at  x  with  a  sufficiently  small  radius  p  >  0  so  that  /2  (Ax  —  b)  <  a 
for  all  x  £  B.  Let  a  =  min{  2.py2 , 5}  €  (0, 1).  We  have  (1  —  a)x  €  B  and  ||(1  —  a)x||i  =  (1  —  a)||x||i  <  ||x||i, 
so  (1  —  a)x  is  both  feasible  and  achieving  an  objective  value  lower  than  the  optimal  one.  Contradiction. 

4)  By  Assumption  3,  we  have  ||x||i  =  r  for  all  x  £  ZT\  otherwise,  there  exists  x  £  ZT  such  that 

r  >  fo(Ax  —  b)  >  inf{||x||i  :  /a (Ax  —  b)  =  /|},  contradicting  Assumption  3.  Since  the  optimal  objective 

fs(Ax  —  b)  is  constant  for  all  x  £  ZT  and  fs  is  strictly  convex  by  Assumption  3,  the  result  follows  from 

Lemma  1. 
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Proof  (Proof  of  Theorem  1  for  problems  (lb)  -(Id) )  This  proof  exploits  Lemma  2.  Since  the  results  of  Lemma 
2  are  are  identical  for  problems  (lb)-(ld),  we  present  the  proof  for  problem  (lb).  The  proofs  for  the  other 
two  problems  are  similar. 

From  Assumption  3,  X\  is  nonempty  so  we  pick  x*  £  X\.  Let  b*  =  Ax* ,  which  is  independent  of  the 
choice  of  x*  according  to  Lemma  2.  We  introduce  the  linear  program 

min||x||i,  s.t .  Ax  =  b* ,  (10) 

and  let  X*  denote  its  solution  set. 

Now,  we  show  that  X\  =  X* .  Since  Ax  =  Ax*  and  ||cc||i  =  ||a:*||i  for  all  x  €  X\  and  conversely 
any  x  obeying  Ax  =  Ax*  and  ||ar||  i  =  ||ai*||i  belongs  to  X\,  it  is  suffices  to  show  that  ||a:||i  =  ||ai*||i 
for  any  x  £  X* .  Assuming  this  does  not  hold,  then  since  problem  (10)  has  a;*  as  a  feasible  solution  and 
has  a  finite  objective,  we  have  a  nonempty  X*  and  there  exists  x  £  X*  satisfying  ||ai|| r  <  ||a:*||1.  But, 
f(Ax  —  b)  =  f(b*  —  b)  =  f(Ax*  —  b)  and  ||x||i  <  ||a:*||i  mean  that  a:  is  a  strictly  better  solution  to  problem 
(lb)  than  x* ,  contradicting  the  assumption  x*  £  X\. 

Since  X\  =  X*,  x*  is  the  unique  solution  to  problem  (lb)  if  and  only  if  it  is  the  same  to  problem  (10). 
Since  problem  (10)  is  in  the  same  form  of  problem  (la),  applying  the  part  of  Theorem  1  for  problem  (la), 
which  is  already  proved,  we  conclude  that  x*  is  the  unique  solution  to  problem  (lb)  if  and  only  if  Condition 
1  holds. 

Proof  (Proof  of  Theorem  3)  The  proof  above  also  serves  the  proof  for  Theorem  3  since  (2a)-(2c)  are  the 
optimality  conditions  of  x*  to  problems  (lb)-(ld),  respectively,  and  furthermore,  given  the  optimality  of  x* , 
Condition  1  is  the  necessary  and  sufficient  condition  for  the  uniqueness  of  x* . 

Remark  1  For  problems  (lb)-(ld),  the  uniqueness  of  a  given  solution  x*  ^  0  is  also  equivalent  to  a  condi¬ 
tion  that  is  slightly  simpler  than  Condition  1.  To  present  the  condition,  consider  the  first-order  optimality 
conditions  (the  KKT  conditions)  (2a)-(2c)  of  x*  to  problems  (lb)-(ld),  respectively,  Given  x*  ^  0,  p  and  v 
can  be  computed.  From  p*  ^  0  it  follows  that  p  >  0.  Moreover,  v  =  0  if  and  only  if  ATX7 f%(Ax*  —  b)  =  0. 
The  condition  below  for  the  case  v  —  0  in  problem  (Id)  reduces  to  Condition  1.  Define 

Pi  =  {i:  \XafVMAx*-  b)\  =  l}, 

P2  =  {i:  | pafXf2(Ax*  -  b)\  =  1}, 

P3  =  {i:\aJXf2(Ax*-b)\  =  u}. 

By  the  definitions  of  3||a;*||i  and  Pi ,  we  have  supp(a;*)  C  P;,  *  =  1,  2,  3. 

Condition  2  Under  the  definitions  I  :=  supp(ai*)  C  P \  and  s  :=  sign(a:J),  matrix  Apt  £  Rmxlpd  0beyS 

1.  submatrix  Aj  has  full  column  rank,  and 

2.  there  exists  y  £  Rm  such  that  Ajy  =  s  and  ||Ap ,yj/||oo  <  1- 

Compared  to  Condition  1,  Condition  2  only  checks  the  submatrix  Api  but  not  the  full  matrix  A. 

It  is  not  difficult  to  show  that  the  linear  programs 

min||x||i,  s.t.  (Ap.)x  =  b* , 

for  *  =  1, 2, 3,  have  the  solution  sets  that  are  equal  to  X*,  Y„,ZT,  respectively.  From  this  argument,  we  have 

Theorem  5  Under  Assumptions  1-3  and  the  additional  condition  that  /i,/2,/3  €E  C1 ,  given  that  x*  ^  0  is 
a  solution  to  problem  (lb),  (lc),  or  (Id),  x*  is  the  unique  solution  if  and  only  if  Condition  2  holds  for  i  =  1, 
2,  or  3,  respectively. 
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Proof  (Proof  of  Corollary  1)  From  Theorem  3,  if  x*  is  the  unique  minimizer  of  problem  (lb)  with  /i(-)  = 
|||  •  HI,  then  Condition  1  holds,  so  there  must  exist  a  vector  y  such  that  Ajy  =  s  and  ||^4/ct/||oo  <  1- 
Combining  with  (3a),  we  have 

A  Ajy  =  As  =  Aj  ( b  —  Aix*). 

Since  Aj  is  a  full  rank  square  matrix,  we  get  y  =  |(6  —  Aix*).  Substituting  this  formula  to  ||.4|’cy||00  <  1, 
we  obtain  condition  (3). 

Proof  (Proof  of  Corollary  2)  The  sufficiency  part  follows  from  Theorem  4.  We  shall  show  the  necessity  part, 
namely,  if  x*  is  the  unique  solution,  then  Aj  has  full  column  rank.  Following  the  assumption  |J|  =  |/|  +  1,  we 
let  {to}  =  J\I.  Since  x*  is  the  unique  solution,  from  Theorem  3,  we  know  that  Aj  has  full  column  rank.  Hence, 
if  Aj  does  not  have  full  column  rank,  then  we  can  have  ai0  =  Ajf3  for  some  (3  £  .  From  Theorem  3,  if  x*  is 

the  unique  minimizer,  then  Condition  1  holds,  and  in  particular,  there  must  exist  a  vector  y  such  that  Aj y  =  s 
and  ll^z/IU  <  1.  Now,  on  one  hand,  as  zo  £  Ic,  we  get  1  >  |a^y|  =  \(3T  Ajy\  =  |/3Ts|;  on  the  other  hand, 
as  zo  €  J,  we  also  have  |a^(6  —  Ax*) \  =  A,  which  implies  1  =  || afo(b—  Ar*)|  =  || f3T  Aj  (b  —  Aa’*)|  =  \/3Ts\, 
where  the  last  equality  follows  from  (2a)  (which  includes  (3a)).  Contradiction. 


3  Recognizing  and  verifying  unique  solutions 

Applying  Theorem  1,  we  can  recognize  the  uniqueness  of  a  given  solution  x*  to  problem  (la)  given  a  dual 
solution  y*  (a  solution  to  problem  (6)).  Specifically,  let  J  :=  {i  :  \ajy*\  =  1},  and  if  Aj  has  full  column 
rank  and  supp(a:*)  =  J,  then  according  to  Theorem  1,  x*  is  the  unique  solution  to  (la).  The  converse  is 
not  true  since  there  generally  exist  multiple  dual  solutions  with  different  J.  The  key  is  to  find  the  one  with 
the  smallest  J.  Several  linear  programming  interior  point  methods  (see  [19]  for  example)  return  the  dual 
solution  y*  with  the  smallest  J,  so  if  either  Aj  is  column-rank  deficient  or  supp(x*)  ^  J,  then  x*  is  surely 
non-unique. 

Corollary  3  Under  Assumption  1,  given  a  pair  of  primal-dual  solutions  ( x*,y *)  to  problem  (la),  let  J  := 
{i  :  \ajy*\  =  1}.  Then,  x*  is  the  unique  solution  to  (la)  if  Aj  has  fidl  column  rank  and  supp(a;*)  =  J.  In 
addition,  if  y*  is  obtained  by  a  linear  programming  interior-point  algorithm,  the  converse  also  holds. 

Similar  results  will  also  hold  for  problems  (lb)-(ld)  if  a  dual  solution  y*  to  (10)  is  available. 

One  can  also  directly  verify  Condition  1.  Given  a  matrix  A  £  Rmxn,  a  set  of  its  columns  indexed  by  J, 
and  a  sign  pattern  s  =  {— 1,1}^,  we  mention  two  approaches  to  verify  Condition  1.  Checking  whether  Aj 
has  full  column  rank  is  straightforward. 

To  check  part  2  of  Condition  1,  the  first  approach  is  to  follow  the  proof  of  Theorem  1.  Note  that 
Condition  1  depends  only  on  A,  /,  and  s,  independent  of  x* .  Therefore,  construct  an  arbitrary  x*  such  that 
supp(x*)  =  /  and  sign(xj)  =  s  and  let  b  =  Ax*.  Solve  problem  (6)  and  let  y*  be  its  solution.  If  y*  satisfies 
part  2  of  Condition  1,  we  are  done;  otherwise,  define  L,  U,  and  t  by  x*  as  in  the  proof,  pick  a  small  a  >  0, 
and  solve  program  (9)  parametrically  in  a  £  [0,b].  The  solution  is  piece-wise  linear  in  a  (it  is  possible  that 
the  solution  does  not  exist  over  certain  intervals  of  a).  Then  check  if  there  is  a  perturbation  to  y*  so  that 
y*  satisfies  part  2  of  Condition  1. 

In  the  second  approach  to  check  part  2  of  Condition  1,  one  can  solve  the  convex  program 

min  -  1°g(1  “  a^y)  +  1°g(1  +  ai  V)’ 

V  i£lc 

s.t.  Ajy  =  s. 


(11) 
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Since  afy  — >  1  or  afy  — >  —1  will  infinitely  increase  the  objective,  (11)  will  return  a  solution  satisfying 
Condition  1,  part  2,  as  long  as  a  solution  exists.  In  fact,  any  feasible  solution  to  (11)  with  a  finite  objective 
satisfies  Condition  1,  part  2.  To  find  a  feasible  solution,  one  can  apply  the  augmented  Lagrangian  method, 
which  does  not  require  Aj  y  =  s  to  hold  at  the  initial  point  (which  must  still  satisfy  \afy\  <  1  for  all*  £  Ic), 
or  one  can  consider  applying  the  alternating  direction  method  of  multipliers  (ADMM)  to  the  equivalent 
problem 

min  -  V'  log(l  -  Zi)  +  log(l  +  zj, 

y.z  z ' 

iei-  (12) 

s.t.  Afy  =  s,  z  —  Ajcy  =  0. 

One  can  start  ADMM  from  the  origin,  and  the  two  subproblems  of  ADMM  have  closed-form  solutions;  in 
particular,  the  2-subproblem  is  separable  in  Zi  s  and  reduces  to  finding  the  zeros  of  3-orcler  polynomials  in 
Zi,  i  £  Ic.  If  (12)  has  a  solution,  ADMM  will  find  one;  otherwise,  it  will  diverge. 

It  is  worth  mentioning  that  one  can  use  alternating  projection  in  [20]  to  generate  test  instances  that  fulfill 
Condition  1. 


4  Conclusions 

This  paper  shows  that  Condition  1,  which  is  previously  known  to  be  sufficient  for  the  solution  uniqueness  of 
the  basis  pursuit  model,  is  also  necessary.  Moreover,  the  condition  applies  to  various  i\  minimization  models. 
The  result  essentially  follows  from  the  fact  that  a  pair  of  feasible  primal-dual  programs  have  strict  comple¬ 
mentary  solutions.  The  result  also  sheds  light  on  numerically  recognizing  unique  solutions  and  verifying 
solution  uniqueness. 
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